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Abstract 

We give conditions under which a function F {t, x, u, ipo, '0) satisfies 
the relation ^ = ^ + §|- • §^ — • ^ along the Pontryagin extremals 
{x{-),u{-),ipQ,ip{-)) of an optimal control problem, where H is the cor- 
responding Hamiltonian. The relation generalizes the well known fact 
that the equality ^ = ^ holds along the extremals of the problem, 
and that in the autonomous case H = constant. As applications of 
the new relation, methods for obtaining conserved quantities along the 
Pontryagin extremals and for characterizing problems possessing given 
constants of the motion are obtained. 

Keywords: dynamic optimization, optimal control, Pontryagin extremals, 
constants of the motion. 



1 Introduction 



A dynamic optimization continuous problem poses the question of what is 
the optimal magnitude of the choice variables, at each point of time, in a 
given interval. To tackle such problems, three major approaches are avail- 
able: dynamic programming; the calculus of variations; and the powerful 
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and insightful optimal control. The calculus of variations is a classical sub- 
ject, born in 1696 with the brachistochrone problem, whose field of appli- 
cability is broadened with optimal control theory. Dynamic programming 
is based on the solution of a partial differential equation, known as the 
Hamilton-Jacobi-Bellman equation, in order to compute a value function. 
Dynamic programming is well designed to deal with optimization problems 
in discrete time. All these techniques are well known in the literature of 
operations research (see e.g. systems theory (see e.g. [|^]), eco- 

nomics (see e.g. |, H] and m, Capi'tulo 14]) and management sciences (see 



e.g ||T^)). Here, we are concerned with the methods and procedures of opti- 
mal control. This approach allows the effective study of many optimization 
problems arising in such fields as engineering, astronautics, mathematics, 
physics, economics, business management and operations research, due to 
its ability to deal with restrictions on the variables and nonsmooth functions 
(see e.g. 0, 0, |7|). 



At the core of optimal control theory is the Pontryagin maximum principle 
- the celebrated first order necessary optimality condition - whose solutions 
are called (Pontryagin) extremals and which are obtained through a function 
H called Hamiltonian, akin to the Lagrangian function used in ordinary cal- 
culus optimization problems (see e.g. |21, |2^)). For autonomous problems 



of optimal control, i.e. when the Hamiltonian H does not depend explicitly 
on time t, a basic property of the Pontryagin extremals is the remarkable 
feature that the corresponding Hamiltonian is constant along the extremals 



(see e.g. [23, |TH|). In classical mechanics this property corresponds to en- 



ergy conservation (see e.g. [|18| , |2j]), while in the calculus of variations it 
corresponds to the second Erdmann necessary optimality condition (see e.g. 
1^). For problems of optimal control that depend upon time t explicitly 
(non-autonomous problems), the property amounts to the fact that the to- 
tal derivative with respect to time of the corresponding Hamiltonian equals 
the partial derivative of the Hamiltonian with respect to time: 

dH OH 

— {t, x(t), n(t), ^0, V'(i)) = ^ {t, x{t),u{t),i;o,m) (1) 

for almost all t (see e.g. p3| , |2|, |l^). This corresponds to the DuBois- 
Reymond necessary condition of the calculus of variations (see e.g. [^). 
Recent applications, in many different contexts of the calculus of variations 
and optimal control, show the fundamental nature of the property (|l|). It 
has been used in ||TT| , |^, ^ to establish Lipschitzian regularity of minimizers; 



in 1 10 1 to establish some existence results; and in ^ to prove some gen- 



eralizations of first Noether's theorem. The techniques used in the proof of 
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the relation are also very useful, and have been applied in contexts far away 
from dynamic optimization (see e.g. [l^). In this note we give conditions 
under which a function x, u, •00) "0) satisfies the equality 

dF _dF ^ dF dH dF dH 

dt dt dx dij) dtp dx ' 

almost everywhere, along the Pontryagin extremals. For F = H equality 
(H) reduces to (|l|). As a corollary, we obtain a necessary and sufficient 
condition for F (t, XjUjipQjip) to be a constant of the motion. From it, one 
is able to find constants of the motion that depend on the control and 
that are not momentum maps, that is, one can find preserved quantities 
F {t, x{t),u{t),tpo,'^i't)) along the Pontryagin extremals {x{-),u{-),ipQ,'>p{-)) 
of the problem, which are not of the form tp{t) ■ C {x{t)). This is in contrast 
with the results obtained in [^, where the conserved quantities are always 
of the form ip[t) ■ C {x{t)). Our condition provides also a method for the 
characterization of optimal control problems with given constants of the 
motion. All these possibilities are illustrated with examples. 

2 Preliminaries 

Without loss of generality (see e.g. Q), we will be considering the opti- 
mal control problems in Lagrange form with fixed initial time a and fixed 
terminal time b {a < b). 

2.1 Formulation of the Optimal Control Problem 

The problem consists of minimize a cost functional of the form 
rb 



J 



[x{-),u{-)] = [ L{t,x{t),u{t))dt, (3) 

J a 

called the performance index, among all the solutions of the vector differen- 
tial equation 

x{t) = (p{t,x{t),u{t)) for a.a. t e [a, 6] . (4) 

The state trajectory x(-) is a n- vector absolutely continuous function 

x(-) e Wi,i([a,6];IR") ; 

and the control u{-) is a r-vector measurable and bounded function satisfying 
the control constraint u{t) G 

n(-) G L^{[a,b];n) . 
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The set C M*" is called the control set. In general, the problem may include 
some boundary conditions and state constrains, but they are not relevant for 
the present study: the results obtained are independent of those restrictions. 
We assume the functions L : [a, 6] x x ^ M and ip : [a, 6] x R" x ^ 
to be continuous on [a,b] x M" x Q and to have continuous derivatives with 
respect to t and x. 



2.2 The Pontryagin Maximum Principle 

We shall now formulate the celebrated Pontryagin maximum principle |22] 



which is a first-order necessary optimality condition. The maximum princi- 
ple provides a generalization of the classical calculus of variations first-order 
necessary optimality conditions and can treat problems in which upper and 
lower bounds are imposed on the control variables - a possibility of consid- 
erable interest in operations research (see 

Theorem 1 (Pontryagin maximum principle). Let (x( ), u( )) be a min- 
imizer of the optimal control problem. Then, there exists a nonzero pair 
{'ipo,'ip{-)) , where tpo <0 is a constant and ip{-) a n-vector absolutely contin- 
uous function with domain [a,b], such that the following hold for almost all 
t on the interval [a,b]: 

(i) the Hamiltonian system 

' x{t) = it,x {t),u{t),i;o,m) 



< 



dip 

dH {t,x{t),u{t),TPo,^{t)) 



= dx 



(ii) the maximality condition 

H {t,x{t),u{t),'il;o,'>p{t)) = max H {t,x{t),v,'ilJo,'>p{t)) ; 

with the Hamiltonian H{t, x, u, ipo,ip) = ipQL{t, x,u) + ^ip ■ ip{t, x, u). 

Definition 1. A quadruple (a;(-), u(-), Vo) V'(')) satisfying the Hamiltonian 
system and the maximality condition is called a (Pontryagin) extremal. 

Remark 1. Different terminology for the function H can be found in the 
literature. The Hamiltonian H is sometimes called "unmaximized Hamilto- 
nian", "pseudo-Hamiltonian" or "Pontryagin function". 



4 



Remark 2. Transversality conditions may also appear in the Pontryagin 
maximum principle. These conditions depend on the specific boundary con- 
ditions under consideration. Our methods do not require the use of such 
transversality conditions and the results obtained are, as already mentioned, 
valid for arbitrary boundary conditions. 

Remark 3. The maximality condition is a static optimization problem. The 
method of solving the optimal control problem via the maximum 

principle consists of finding the solutions of the Hamiltonian system by the 
elimination of the control with the aid of the maximality condition. The 
required optimal solutions are found among these extremals. 

The proof of the following theorem can be found, for example, in ^. 

Theorem 2. If {x{■),u{■),^l'o,t|J{■)) is a Pontryagin extremal, then the func- 
tion H (t,x{t),u{t),ipQ,ilj{t)) is an absolutely continuous function oft and 
satisfies the equality where on the left-hand side we have the total deriva- 
tive with respect to t, and on the right-hand side the partial derivative of the 
Hamiltonian with respect to t. 

As a particular case of Theorem ||, when the Hamiltonian does not depend 
explicitly on t, that is when the optimal control problem is autonomous - 
functions L and ip do not depend on t - then the value of the Hamiltonian 
evaluated along an arbitrary Pontryagin extremal {x{-),u{-),jpo,xp{-)) of the 
problem turns out to be constant: 

-i/'O) "0(0) = const , t ^ [a,b] . 

We remark that Theorem |2| is a consequence of the Pontryagin maximum 
principle. We shall generalize Theorem Q in Section ^ Before, we review 
some facts from functional analysis needed in the proof of our result. 

2.3 Facts from Functional Analysis 

First we introduce the concept of an absolutely continuous function in t 
uniformly with respect to s. 

Definition 2. Let (j){s,t) be a real valued function defined on [a,b] x [o, 6]. 
The function (f>{s, t) is said to be an absolutely continuous function in t 
uniformly with respect to s if, given e > 0, there exists 6 > 0, independent 
of s, such that for every finite collection of disjoint intervals (aj, bj) C [a, b] 

(bj -aj)<5^Y^ \(j){s, bj) - (l){s, aj)\ <£ (s G [a, b]) . 
j j 
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The proof of the fohowing two propositions can be found in jl^, p. 74]. 

Proposition 3. Let F [t, x,u,'iIjo,'iIj) , F : [a,b] x W x Q x Rq x W ^ R, 
be continuously differentiable with respect to t, x, ^ for u fixed, and assume 
that there exists a function G{-) € Li {[a,b];R) such that 

\\V^t,x,^)F{t,x{t),u{s),iPo,^P{t))\\ <G{t) {s,te [a,b]) . 

Then (j){s,t) = F {t, x{t),u{s),ipQ,ilj{t)) is absolutely continuous in t uni- 
formly with respect to s on [a,b]. 

Proposition 4. Let (j){s, t), (f) : [a, b] x [a, b] — > R, be an absolutely continuous 
function in t uniformly with respect to s satisfying 

(f){t, t) = max (f){s, t) 

s&[a,b\ 

in a set dense in [a, 6]. Then the function (j){t,t) can be uniquely extended 
to a function m{t) absolutely continuous on [a,b]. 

3 Main Result 

Our result is a generalization of the Theorem 

Theorem 5. If F (t, x, u, "00 1 "0) ^-5 o, real valued function as in Proposition ^ 
and besides satisfies 

F {t,x{t),u{t),iPo,xl;{t)) =maxF {t,x{t),v,iPo,Ht)) (5) 

a.e. in t G [a,b] along any Pontryagin extremal (x(-), n(-), -00) V'(')) of the 
optimal control problem, then t — > F {t, x{t) , u{t) , ■ip{t)) is absolutely con- 
tinuous and the equality 

dF _ dF dF dH dF dH 

dt dt dx 5-0 dil) dx 

holds along the extremals. 

Proof. Our proof is an extension of the standard proof of Theorem |^. Let 
(x(-), u(-), 00) V'(')) be a Pontryagin extremal of the problem. Setting v = 
u{s) in (|5|) we obtain that (j){s,t) = F (t, x{t) , u{s) , 'ipo, ip{t)) satisfies 

0(t,t) > 0(s,t), se[a,b], (7) 
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for t in a set of full measure on [a, b]. Proposition ^ then implies that m{t) = 
(j){t,t) = F {t^x{t),u{t),il)Q^ilj{t)) is an absolutely continuous function on 
[a, h]. It remains to prove that 

. , \ dF , . , , dF , . , . dFl , ... dF , ... dFl , ... 

where 7r(i) = {t,x{t),u{t),ipo,'4'{t)). Since 

m(i + /i) - m(t) _ (j)(t + h,t + h)- cpit, t + /i) t + h) - (j){t, t) 

h h h 

and by the hypotheses the left-hand side and the second term on the right- 
hand side have a limit as /i — > 0, one concludes that the first term on the 
right must have a limit as well. From (Q) (j){t + h,t + h) > (j){t, t + h) and it 
follows that '?^(*+^'*+^^~'^(*'*+^) nonnegative when h > and nonpositive 
when h < 0; thus, its limit must be zero when h ^ 0. In this way we obtain 
that 

. y F{t + h, xjt + h),u{t),2l;o,^lJ{t + h)) - F {t, V^p, V>(0) 

mit) = lim ; 

h^o h 

dF dF OF 

= ^ ^^^^^^ + h ^^^^^^ ■ + h ^""^^^^ ■ "^^^^ ' 

and the conclusion follows from the Hamiltonian system. □ 

Corollary 6. Let F {t,x,u,ipo,ip), F : [a, 6] x x i7 x Mg x ^ 

M, be continuously differentiable with respect to t, x, ijj for u fixed; and 
{x{-),u{-),ipo,ip{-)) be an extremal. If 

(i) F {t,x{t),u{t),ipQ,'ilj{t)) is absolutely continuous int; 

(ii) F (t, -00) "0(0) = ^(0) ^) ^o> V'l*)) o,-^- ina<t<b; 
then the equality holds along the extremal. 

Possible applications of Theorem follow in the next section. 



4 Applications of the Main Result 

Solving the Hamiltonian system by the elimination of the control with the 
aid of the maximality condition is typically a difficult task. Therefore, it is 
worthwhile to look for circumstances which make the solution easier. This 
is the case when the extremals don't change the value of a given function. 
Indeed, the existence of such a function, called constant of the motion, may 
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be used for reducing the dimension of the Hamiltonian system (see e.g. p^ , 
Modulo 5]). In extreme cases, with a sufficiently large number of (indepen- 
dent) constants of the motion, one can solve the problem completely. 

4.1 Constants of the Motion 

From Theorem ^ one immediately obtains a necessary and sufficient condi- 
tion for a function to be a constant of the motion. 

Definition 3. A quantity F(t, x, u, "001 V') which is constant along every 
Pontryagin extremal {x{-),u{-),ipo,'ip{-)) of the problem, is called a constant 
of the motion. 

Corollary 7. Under the conditions of Theorem^, -F(t, x, u, -00; "0) ^•s o, con- 
stant of the motion if and only if 

dF dF dH dF dH _ ^ 

dt dx dtp dip dx 
holds, almost everywhere, along the Pontryagin extremals of the optimal 
control problem. 

Example 1. (n = 4, r = 2, 17 = M^) Let us consider the problem 



({ui{t)f + {u2{t)f^ dt — ^min, 



xi{t) = X3{t) 

X2{t) = X4{t) 

X3{t) = -Xi{t) Uxi{t)f + {X2{t)f) + Ui{t) 

^X4(t) = -X2{t) ((Xi(t))2 + ix2{t)f) + U2it) . 

The corresponding Hamiltonian function is 

H {xi,X2,X3,X4,Ui,U2,1p0,'4>l,1p2,'4'3,1p4.) = IpO {uj + U^) + 1P1X3 

+ tp2X4: - 1P3X1 (xf + xl) + 1P3U1 - 1P4X2 {xf + xl) + 1piU2 . 

We claim that 

F = -1plX2 + 1P2X1 - 1P3XA + tp4:X3 (9) 



is a constant of the motion for the problem. Direct calculations show that 
dF dF dH dF dH _ ^ ^ ^ ^ 



From the maximality condition it follows that = and = 0, that is, 
2ipQUi + ip3 = ^ and 2ipQU2 + '04 = 0. Using these last two identities in ( p^ ) 
one concludes from Corollary ^ that (^) is a constant of the motion. 
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4.2 Characterization of Optimal Control Problems 



We shall endeavor here to find a method to synthesize optimal control prob- 
lems with given constants of the motion. If a function F is fixed a priori, 
we can regard equality (^) as a partial differential equation in the unknown 
Hamiltonian H. Obviously, if this differential equation admits a solution, 
then an optimal control problem can be constructed with the constant of 
the motion F. We shall illustrate the general idea in special situations. 



Example 2. The Hamiltonian H is a constant of the motion if and only if 

dH 
at 



^M- = 0. Condition is trivially satisfied for autonomous problems. 



±{t) 



Example 3. Function il^x + Ht is a constant of the motion if and only if 
H = — ~ ^BT^' Condition is satisfied, for example, for problems of 
the form (0 < a < 6) 

L{tx{t),u{t)) , 
— ^ — ^-—--^dt — > min , 
t 

ip{tx{t),u{t)) 

Example 4. We conclude from Corollary ^ that a necessary and sufficient 
condition for Hipx to be a constant of the motion is 

, dH ^BH dH 
ipx— + WH— - Hx— = . 
at dip ox 

A simple problem with constant of the motion Hipx is therefore 
/ L{u{t))dt — s- min , 

J a 

X{t) = if {u{t)) x{t) . 

Example 5. The following optimization problem is important in the study 
of cubic polynomials on Riemannian manifolds (see |^, p. 39] and p6[| ). 
Here we consider the particular case when one has 2-dimensional state and 
n controls: 
f-T 

(iuiit)f + ■■■ + iun{t)f) dt min , (11) 
^ ^ 

(xi{t) = X2{t) , 

\x2it) = Xi (Xi(t)) Ui{t) + ---+Xn (Xiit)) Unit) . 

Functions Xi{-), i = 1, n, are assumed smooth. The Hamiltonian for 
the problem is 

H = lpo{u1-\ \-ul) + i>iX2 + 1p2 {Xi{xi)ui H h Xn{xi)Un) ■ 
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As far as the problem is autonomous, the Hamiltonian is a constant of the 
motion. We are interested in finding a new constant of the motion for the 
problem. We will look for one of the form 

F = kilpiXi + k2i'2X2 , 

where ki and k2 are constants. This is a typical constant of the motion, 
known in the literature by momentum map (see Q). First we note that 

dF dF , BF , ^ dF ^ OF , 
-wr = 0, ^— = kiipi , -— = k2t2 , TT— = kixi , — — = ^2X2 , 

Ot 0X1 0X2 Olpi Oip2 

and 

dH dH 

- — = ip2 (X[{xi)ui H h X'^{xi)un) , t; — = ipi , 

OXi 0x2 

dH ^ . . ^ , , 

77— = X2 , 77— = Ai(xi)ni H h Xn[Xi)Un ■ 

dipi dtp2 
Substituting these quantities into (^) we obtain that 

kl1plX2 + k2lp2 iXl{xi)ui H h Xn{xi)Un) 

- kiXi'4)2 {X[{xi)Ui H h X'^{xi)Un) - k2X2'4'l = . 

The equality is trivially satisfied if ki = k2 and X[{xi)xi = Xj(xi), i = 
1, . . . , n. We have just proved the following proposition. 

Proposition 8. // the homogeneity condition Xi (Axi) = \Xi{xi) (i = 
1, . . . , n), VA > 0, holds, then 7pi(t)xi{t)+ijj2(t)x2{t) is constant int € [0,T] 
along the extremals of the problem (|lT|) . 
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